On-line video filtering

ABSTRACT

Some embodiments relate to a system and method to increase the speed of a computer determination whether a video contains a particular content. In some embodiments, the quantity of data in the video is first reduced while preserving the searched-for content. Optionally, first, the size of the data is reduced by reducing the resolution, for example resolution may be reduced without searching and/or processing the full data set. Additionally or alternatively, low quality and/or empty data is removed from the dataset. Additionally or alternatively, redundant data may be searched out and/or removed. Optionally, after data reduction, the reduced dataset is analyzed to determine if it contains the searched-for content. Optionally, an estimate is made of the probability of the full dataset containing the searched-for content.

RELATED APPLICATION/S

This application is a continuation in part of U.S. patent applicationSer. No. 16/666,440 filed 29 Oct. 2019 which is a continuation in partof PCT/IL2019/051142 filed 23 Oct. 2019 which claims priority benefitfrom U.S. Provisional Application No. 62/758,589, filed Nov. 11, 2018.

This application claims the benefit of priority of U.S. patentapplication Ser. No. 16/666,440 filed 29 Oct. 2019 and of InternationalPatent Application no. PCT/IL2019/051142 filed 23 Oct. 2019

This application claims the benefit of priority under 35 USC § 119(e) ofU.S. Provisional Patent Application No. 62/758,589, filed Nov. 11, 2018,the contents of all of the above are incorporated herein by reference intheir entirety.

FIELD AND BACKGROUND OF THE INVENTION

The present invention, in some embodiments thereof, relates to ascreening videos and, more particularly, but not exclusively, to a fastmethod of detecting a content in a video.

U.S. Pat. No. 8,798,402 apparently discloses “A system and method foranalyzing video include segmenting video stored in computer readablestorage media into keyframes. Near-duplicate keyframes are representedas a sequence of indices. The near-duplicate keyframes are rendered in agraphical representation to determine relationships between videocontent.”

U.S. Pat. No. 9,805,280 appears to the present inventor disclose, “Asystem including: (a) a network hub or port adapted to detect imagefiles in transit according to their file designations; (b) an objectdetector configured to identify one or more regions of interest (ROI) ineach image file as potentially containing an object of interest (OOI);(c) a feature analyzer adapted to express one or more GeneralClassification Features (GCF) of each ROI as a vector; and (d) adecision module adapted accept or reject each ROI as containing said OOIbased upon the one or more GCF vectors.”

Additional background art includes US published patent application no.20040170321, U.S. Pat. Nos. 8,798,400, 8,311,344, US published patentapplication no. US published patent application no. 20120148149, USpublished patent application no. 20160358628, US published patentapplication no. 20160092561, US published patent application no.20110301447, U.S. Pat. Nos. 6,751,354, 6,404,925, and 5,740,270.

SUMMARY OF THE INVENTION

According to an aspect of some embodiments of the invention, there isprovided a method to determine a presence of a sought content in videoincluding: summarizing the video to form a summary; preserving thecontent of the video in the summary; and analyzing the summary for thepresence of the sought content.

According to some embodiments of the invention, the summarizingincludes: determining a representative element volume (REV) of thesought content and reducing a level of precision to an order of the REV.

According to some embodiments of the invention, reducing is to a levelof precision that is inherent in the video.

According to some embodiments of the invention, the summarizing includesselecting a collection of key frames and the summary includes thecollection of key frames.

According to some embodiments of the invention, the summarizing includesremoving low information data.

According to some embodiments of the invention, removing low informationdata includes computing an entropy of a portion of the video andremoving the portion of the video when the entropy is low.

According to some embodiments of the invention, removing low informationdata includes computing an entropy of a key frame and removing the keyframe in response to the computed entropy being low.

According to some embodiments of the invention, the summarizing includesremoving redundant data.

According to some embodiments of the invention, removing redundant dataincludes computing a General Classification Features (GCF) value of atleast two frames and comparing the GCF between the at least two framesand removing at least one of the at least two frames when the GCF of theat least two frames is similar.

According to some embodiments of the invention, the computing a GCFincludes computing an Edge Histogram value for each of the at least twoframes.

According to some embodiments of the invention, the comparing is of allframes in a group without accounting for sequencing within the group.

According to some embodiments of the invention, the method furtherincludes determining an uncertainty of a determination of the presenceof the content in the video.

According to some embodiments of the invention, the determining anuncertainty of a presence of the content in the video includes computingan uncertainty of determination the presence in a sample.

According to some embodiments of the invention, the determining anuncertainty of the presence of the content in the video includescomputing an uncertainty of the presence of the content between twosamples.

According to some embodiments of the invention, the uncertainty of thepresence of the content between two samples takes into account a spatialautocovariance.

According to some embodiments of the invention, the method furtherincludes performing further processing when the uncertainty is high.

According to an aspect of some embodiments of the invention, there isprovided a system for in-line content filtering of video including: aconnection to a public network a server receiving video content from apublic network over the connection; the server including a video summarymodule configured to output a summary of the video content, the summarybeing smaller than the video content and preserving a sought content, adetection module configured to detect a sought content in the summary, adecision module configured for facilitating sending the video content auser device in response to the detection module not detecting the soughtcontent and to inhibit the sending when the detection module detects thesought content in the summary a connection between the server and a userdevice configured for sending the video content to the user device inaccordance with a decision from the decision module.

According to some embodiments of the invention, the server furtherincludes a video disassembly module configured to separate key framesfrom the video and supply them to the video summary module and whereinthe summary includes a subset of the key frames.

According to some embodiments of the invention, the video summary moduleis configured for: determining a representative element volume (REV) ofthe sought content and reducing a level of precision the summary to anorder of the REV.

According to some embodiments of the invention, the video summary moduleis configured for selecting a collection of key frames and the summaryincludes the collection of key frames.

According to some embodiments of the invention, the video summary moduleis configured for removing low information data.

According to some embodiments of the invention, the video summary moduleis configured for computing an entropy of a portion of the video andremoving the portion of the video when the entropy is low.

According to some embodiments of the invention, the video summary moduleis configured for computing an entropy of a key frame and removing thekey frame from the summary in response to the computed entropy beinglow.

According to some embodiments of the invention, the video summary moduleis configured for removing redundant data.

According to some embodiments of the invention, the video summary moduleis configured for computing a General Classification Features (GCF)value of at least two frames and comparing the GCF between the at leasttwo frames and removing at least one of the at least two frames when theGCF of the at least two frames is similar.

According to some embodiments of the invention, the video summary moduleis configured for computing an Edge Histogram value for each of the atleast two frames and wherein a value of the GCF depends on the EdgeHistogram.

According to some embodiments of the invention, the video summary moduleis configured for comparing all frames in a group without accounting forsequencing within the group.

According to some embodiments of the invention, the server is furtherconfigured for determining an uncertainty of a determination of apresence of the content in the video.

According to some embodiments of the invention, the video summary moduleis configured for computing an uncertainty of determination the presencein a sample.

According to some embodiments of the invention, the video summary moduleis configured for computing an uncertainty of the presence of thecontent between two samples.

According to some embodiments of the invention, the video summary moduleis configured for computing a spatial autocovariance between frames.

According to some embodiments of the invention, the video summary moduleis configured for performing further processing when the uncertainty ishigh.

Unless otherwise defined, all technical and/or scientific terms usedherein have the same meaning as commonly understood by one of ordinaryskill in the art to which the invention pertains. Although methods andmaterials similar or equivalent to those described herein can be used inthe practice or testing of embodiments of the invention, exemplarymethods and/or materials are described below. In case of conflict, thepatent specification, including definitions, will control. In addition,the materials, methods, and examples are illustrative only and are notintended to be necessarily limiting.

As will be appreciated by one skilled in the art, some embodiments ofthe present invention may be embodied as a system, method or computerprogram product. Accordingly, some embodiments of the present inventionmay take the form of an entirely hardware embodiment, an entirelysoftware embodiment (including firmware, resident software, micro-code,etc.) or an embodiment combining software and hardware aspects that mayall generally be referred to herein as a “circuit,” “module” or“system.” Furthermore, some embodiments of the present invention maytake the form of a computer program product embodied in one or morecomputer readable medium(s) having computer readable program codeembodied thereon. Implementation of the method and/or system of someembodiments of the invention can involve performing and/or completingselected tasks manually, automatically, or a combination thereof.Moreover, according to actual instrumentation and equipment of someembodiments of the method and/or system of the invention, severalselected tasks could be implemented by hardware, by software or byfirmware and/or by a combination thereof, e.g., using an operatingsystem.

For example, hardware for performing selected tasks according to someembodiments of the invention could be implemented as a chip or acircuit. As software, selected tasks according to some embodiments ofthe invention could be implemented as a plurality of softwareinstructions being executed by a computer using any suitable operatingsystem. In an exemplary embodiment of the invention, one or more tasksaccording to some exemplary embodiments of method and/or system asdescribed herein are performed by a data processor, such as a computingplatform for executing a plurality of instructions. Optionally, the dataprocessor includes a volatile memory for storing instructions and/ordata and/or a non-volatile storage, for example, a magnetic hard-diskand/or removable media, for storing instructions and/or data.Optionally, a network connection is provided as well. A display and/or auser input device such as a keyboard or mouse are optionally provided aswell.

Any combination of one or more computer readable medium(s) may beutilized for some embodiments of the invention. The computer readablemedium may be a computer readable signal medium or a computer readablestorage medium. A computer readable storage medium may be, for example,but not limited to, an electronic, magnetic, optical, electromagnetic,infrared, or semiconductor system, apparatus, or device, or any suitablecombination of the foregoing. More specific examples (a non-exhaustivelist) of the computer readable storage medium would include thefollowing: an electrical connection having one or more wires, a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), an optical fiber, a portable compact disc read-onlymemory (CD-ROM), an optical storage device, a magnetic storage device,or any suitable combination of the foregoing. In the context of thisdocument, a computer readable storage medium may be any tangible mediumthat can contain, or store a program for use by or in connection with aninstruction execution system, apparatus, or device.

A computer readable signal medium may include a propagated data signalwith computer readable program code embodied therein, for example, inbaseband or as part of a carrier wave. Such a propagated signal may takeany of a variety of forms, including, but not limited to,electro-magnetic, optical, or any suitable combination thereof. Acomputer readable signal medium may be any computer readable medium thatis not a computer readable storage medium and that can communicate,propagate, or transport a program for use by or in connection with aninstruction execution system, apparatus, or device.

Program code embodied on a computer readable medium and/or data usedthereby may be transmitted using any appropriate medium, including butnot limited to wireless, wireline, optical fiber cable, RF, etc., or anysuitable combination of the foregoing.

Computer program code for carrying out operations for some embodimentsof the present invention may be written in any combination of one ormore programming languages, including an object oriented programminglanguage such as Java, Smalltalk, C++ or the like and conventionalprocedural programming languages, such as the “C” programming languageor similar programming languages. The program code may execute entirelyon the user's computer, partly on the user's computer, as a stand-alonesoftware package, partly on the user's computer and partly on a remotecomputer or entirely on the remote computer or server. In the latterscenario, the remote computer may be connected to the user's computerthrough any type of network, including a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

Some embodiments of the present invention may be described below withreference to flowchart illustrations and/or block diagrams of methods,apparatus (systems) and computer program products according toembodiments of the invention. It will be understood that each block ofthe flowchart illustrations and/or block diagrams, and combinations ofblocks in the flowchart illustrations and/or block diagrams, can beimplemented by computer program instructions. These computer programinstructions may be provided to a processor of a general purposecomputer, special purpose computer, or other programmable dataprocessing apparatus to produce a machine, such that the instructions,which execute via the processor of the computer or other programmabledata processing apparatus, create means for implementing thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

These computer program instructions may also be stored in a computerreadable medium that can direct a computer, other programmable dataprocessing apparatus, or other devices to function in a particularmanner, such that the instructions stored in the computer readablemedium produce an article of manufacture including instructions whichimplement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer,other programmable data processing apparatus, or other devices to causea series of operational steps to be performed on the computer, otherprogrammable apparatus or other devices to produce a computerimplemented process such that the instructions which execute on thecomputer or other programmable apparatus provide processes forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks.

Some of the methods described herein are generally designed only for useby a computer, and may not be feasible or practical for performingpurely manually, by a human expert. A human expert who wanted tomanually perform similar tasks might be expected to use completelydifferent methods, e.g., making use of expert knowledge and/or thepattern recognition capabilities of the human brain, which would bevastly more efficient than manually going through the steps of themethods described herein.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S)

Some embodiments of the invention are herein described, by way ofexample only, with reference to the accompanying drawings. With specificreference now to the drawings in detail, it is stressed that theparticulars shown are by way of example and for purposes of illustrativediscussion of embodiments of the invention. In this regard, thedescription taken with the drawings makes apparent to those skilled inthe art how embodiments of the invention may be practiced.

In the drawings:

FIG. 1 is a flow chart illustration of on line filtering of a video inaccordance with an embodiment of the current invention;

FIG. 2A is a schematic illustration of a raw video in accordance with anembodiment of the current invention in a schematic illustration of datareduction of a video of FIGS. 2A-2E in accordance with an embodiment ofthe current invention;

FIG. 2B is a schematic illustration of a set of video frames inaccordance with an embodiment of the current invention;

FIG. 2C is a schematic illustration of a set of video frames afterremoval of null frames in accordance with an embodiment of the currentinvention;

FIG. 2D is a schematic illustration of a set of video frames afterremoval of redundant frames in accordance with an embodiment of thecurrent invention;

FIG. 2E is a schematic illustration of a set of video frames afterremoval of non-candidate frames in accordance with an embodiment of thecurrent invention;

FIG. 3 is a block diagram illustration of a system of on-line videofiltering in accordance with an embodiment of the current invention; and

FIG. 4 is a flow chart illustration of determining a content of a videoin accordance with an embodiment of the current invention.

DESCRIPTION OF SPECIFIC EMBODIMENTS OF THE INVENTION

The present invention, in some embodiments thereof, relates to anon-line filtering of videos and, more particularly, but not exclusively,to a fast method of detecting a content in a video.

Overview

In some embodiments, there may be a desire to reduce the time and/orincrease accuracy for analyzing a video for a content. For example, foron-line content filtering there is a desire to analyze content(supplying permissible content and/or removing undesired content) and/orsupply the content to a user with a small delay time, for example ofbetween a fraction of second to a second to a few seconds. Such fastanalysis may be difficult, especially when dealing with large dataobjects, for example video content and/or fine distinctions betweenpermissible and impermissible content (for example the different betweenpermissible sports videos and undesired pornography and/or violence). Insome embodiments, the speed of analysis will be improved by makingrelatively fast data transformations before analyzing for content. Insome embodiments, the purpose of the data transformation may be toreduce the total computational burden. For example, a fasttransformation may be used to a reduce data before a more expensivetransformation. In some embodiments, removal of redundant and/or uncleardata when forming the summary may reduce identification errors that mayresult from analyzing noisy data and/or from random errors that come upwhile analyzing the large data set and/or from bias of repeated frames.

An aspect of some embodiments of the current invention relates to asystem and method to increase the speed of a computer determinationwhether a video contains a particular content. In some embodiments, thequantity of data in the video is first reduced while preserving thesearched-for content. The reduction is configured both for efficiency(e.g. high-speed execution) and to avoid losing the searched-forcontent. Optionally, first, the size of the data is reduced quickly, byreducing the resolution, for example resolution may be reduced withoutsearching and/or processing the full data set. Additionally oralternatively, low quality and/or empty data is removed from thedataset. Additionally or alternatively, redundant data may be searchedout and/or removed. Optionally, after data reduction, the reduceddataset is analyzed to determine if it contains the searched-forcontent. Optionally, part of video may be reduced to a collection ofimages. Images may optionally may be analyzed using image analysistools, an example of such tools is disclosed in U.S. Pat. No. 9,805,280of the present inventor. Optionally, based on the results of theanalysis of the reduced dataset and/or knowledge about the removed data,an estimate is made of the probability of the full dataset containingthe searched-for content.

In some embodiments resolution of a video is reduced. For example, withrespect to the searched-for content, a representative elementary volume(REV) is defined. The REV may be in time and/or area. For example, whenscanning a film for pornographic images it may be decided thatsignificant pornographic scenes are unlikely to last less than 5seconds. Then a REV (Representative elementary volume) may be defined as5 seconds. Then the time resolution of the dataset may safely be reducedto something of the order of the REV, for example between 1 to 2 REV'sand/or between ½ to 1 REV and/or between ¼ to ½ REV and/or less than ¼REV. Optionally, the resolution reduction will be to a level that iseasy to reach. For example, for a digital video it may be easy to reduceto the time resolution of the video by keeping only key frames andignoring transition data. For example, if the frame interval is lessthan the REV then a reduced video may include only a portion of the keyframes.

In some embodiments, a video content scheme may remove low informationdata from a data set before analyzing the for content. For example, alow-cost algorithm may be used to detect low quality and/or lowinformation content sections.

For example, a scalar and/or vector value may be assigned to each framein a frame set. For example, key frames may be sorted according to theirentropy levels. Low entropy frames may have little information and/ormay be discarded. Optionally an easily computed and/or compared quantitymay be used to indicate the information level.

In some embodiments, redundant data may be removed before analysis.Optionally, the redundant data will be found using an inexpensive quickroutine without requiring high degree of quality in the identification.For example, a scalar and/or vector quantity may be used to characterizeeach key frame. For example, using a General Classification Function(GCF) for example an Edge Histogram Distribution (EHD).

In some embodiments, temporal data and/or sequencing of images may beignored. For example, when removing redundant frames and/or detectingcontent temporal information may be ignored. Alternatively oradditionally, frames be grouped in sets of a short time span but precisesequencing may be ignored. Alternatively or additionally, routines mayconsider sequencing and/or movement.

An aspect of some embodiments of the current invention relates toevaluating an uncertainty in a determination of a content of a videobased on a quick analysis of an abbreviated version of the video. Insome embodiments, after analyzing a video for a sought content aprocessor will estimate a probability that the content was missed and/ormiss-identified. For example, the estimated probability will take intoaccount the size of the sampling interval and the size of the REV and/orthe uncertainty in analysis of each sample and/or the association withcontent found in the sample with the sought content and/or rate ofchange. Optionally, sample may be sent for further testing based on theeffect of improving analysis of that sample on level of uncertainty inthe entire video.

Specific Embodiments

Before explaining at least one embodiment of the invention in detail, itis to be understood that the invention is not necessarily limited in itsapplication to the details of construction and the arrangement of thecomponents and/or methods set forth in the following description and/orillustrated in the drawings and/or the Examples. The invention iscapable of other embodiments or of being practiced or carried out invarious ways.

FIG. 1 is a flow chart illustration of on-line filtering of a video inaccordance with an embodiment of the current invention. In someembodiments, content delivered to a user is monitored 102 for undesiredvideo content. For example, a filtering server may intercept 104 videodata and summarize 106 the video. The resulting summary is optionallyanalyzed 108 for example to determine if it includes an undesiredcontent. For example, a search algorithm seeks the content in thesummary. In some embodiments, the server will determine 110 a probablyof the undesired content searched for in the summary is present in therest of the video that were not included in the search and/or includingthe probability that content present in the summary was not detected.Optionally, the server may compare the reduction in uncertainty ofpresence of undesired content in the video from reducing uncertainty ineach of at least two samples and/or a select 112 a frame. For example,it may be determined which frame to send for further processing 114 tobest reduce uncertainty. For example, the further processing 114 mayinclude manual analysis of a set of one or more frames. For example, forvideos, when the probability of infection with undesired content and/orthe quantity and/or the quality of the undesired content is not beyondan acceptable limit, the server may allow delivery 118 of the content toa user. For example, for videos, when the probability of infection withundesired content and/or the quantity and/or the quality of theundesired content is beyond an acceptable limit, the server may block120 delivery of the content to a user.

In some embodiments, the summarizing 106of the data is arranged tofacilitate fast of forming the summary and/or fast analyses of thesummary for the content. Alternatively or additionally, the summarizing106 of the data is arranged to avoiding removing significant data and/orreduce errors in the analysis 108 for content.

FIGS. 2A-2E are a schematic illustrations of data reduction of a videoin accordance with an embodiment of the current invention. FIG. 2Aillustrates a raw video with a set of frames 230 a-230 d and transitiondata 232. Optionally redundant data will be removed leaving enough datato determine the presence of the searched-for content with reducedexcess data that might slow down analysis of content.

FIG. 2A illustrates reducing video is from the raw video 204 byextracting 234 frames 230 a-230 d to give a set 222 of frames 230 a-230d (e.g. as illustrated in FIG. 2B). For example, the raw video mayinclude a sequence of key frames 230 a-230 d and/or transitioninformation 232 between the frames 230 a-230 d. The summary may be basedon reducing temporal resolution, for example by extracting 234 and/orusing only the key frames 230 a-230 d (removing the transitioninformation 232 and/or avoiding decoding the transition data 232).Alternatively or additionally, for example where the temporal REV of thesearched-for content is greater than an interval between key frames 230a-230 d, some of the key frames 230 a-230 d may be removed duringextraction 234 of frames, further reducing the temporal resolution. Forexample, a frame may occur every two seconds while a REV is 5 seconds.Accordingly, every other key frame may be removed and/or one key framemay be retained in every 4 second period. Alternatively or additionally,the level of spatial resolution in the key frames may be reduced. Forexample, key frames 230 a-230 d may be compressed using known picturecompression means and/or the resolution may be reduced by combiningpixels.

In some embodiments, the frame set 222 may be further summarized to areduced frame set 224 (e.g. as illustrated in FIG. 2C). For example,null frames containing little valuable information may be removed 236.For example, a frame 230 b having low entropy may be removed 236.Alternatively, null frames 230 may be removed from the full frame set222 and then excess frames may be removed to reduce temporal resolution.In some embodiments, removal of transition data 232 and/or poorlydefined frames may also reduce errors in the analysis of content. Forexample, analyzing blurry and/or poorly defined inter-frame data and/orlow entropy blurry frames may add to the likelihood of erroneousconclusions without significantly improving the analysis for realcontent. Thus, removing this data may increase the accuracy of theanalysis. For example, the removal of interframe data 232 and/or lowentropy data (e.g. null frame 230 b) may serve a function like the humanmind's filtering of extraneous background.

In some embodiments, the reduced frame set 224 may be further summarizedby removing 238 redundant frames (e.g. frame 230 d) resulting in aunique frame set (e.g. set 226 of FIG. 2D). Optionally, a contentdependent routine may further reduce the unique from set 226 byeliminating 240 those frames that can easily be determined as unlikelyto contain the searched-for content. A summary may result in a smallsample preserved data set 228 which will be searched for thesearched-for content. In some embodiments, removal of transition data232 and/or poorly defined frames 230 b and/or redundant frames 230 d mayalso reduce errors in the analysis of content. For example, analyzingthe same frame multiple times may increase the chance of incorrectlyidentifying a content without significantly adding to the accuracy ofthe analysis. Furthermore, the presence of a lot of repeated slides maybias the data. Thus, in some embodiments removing 238 redundant data 230d may increase the accuracy of the analysis.

In some embodiments, the order of the operations of summarizing thedataset is from the simple to the more difficult. For example, the fullvideo data set 204 may be reduced by extracting 234 key frames 230 a-230d. Extracting 234 key frames 230 a-230 d optionally requires no decisionmaking and/or analyzing of the data, the key frames 230 a-230 d areessentially separate from the transition data 232 in the video file. Forexample, a further step may include removing 236 low information frames230 b. To recognize a low information frame 230 b may include computinga simple function on individual frames (e.g. entropy) without findingrelations between of different frames. Decisions may optionally be madeby comparing a simple value (for example a scalar entropy value) to aconstant threshold value. Optionally, as the data set becomes smallermore complex operations are used for further reduction. For example, onthe reduced frame set 224, redundant frames may be removed 238 includingcomparing slides among the entire set 224 (e.g. frame 230 a, 230 c and230 d) and/or comparing slides among subsets and/or in accordance with asequence of the frames, for example to find and/or remove 238 redundantsimilar frames (e.g. removing 238 frame 230 d which is similar to frame230 a which is kept) resulting in a further reduced unique frame set226. For example, the unique frame set 226 may be further processedusing more complex content recognition routines (for example GCFfunctions, flesh recognition routines and the like) to get a samplepreserved set 228 for full analysis. Comparing frames may includecomputing a distance between scalar and/or vector quantities the measureone or more aspects of different frames.

In some embodiments, a GCF function (for example EHD) may be used forone of both analyzing the content of a “frame” (e.g. deciding in theframe contains significant information and/or if the frame includes aparticular content) and/or for identifying a frame signature (forexample to compare frames and/or remove redundant frames). Theidentifying of a signature is optionally flexible enough to recognizeredundant frames even when there are slight changes within the frames.Identifying a signature is optionally very fast. In some embodiments,when identifying a signature, the GCF output may be converted from avector to a scalar (for example, by connecting all the parameters), forexample to more easily and/or more quickly identify similar “frames.”

In some exemplary embodiments of the invention, an initial screening torecognize and/or remove 240 non-candidates is conducted using one ormore Boosted Cascade classifier algorithms selected from the groupconsisting of Haar, LBP, LRD, LRP, HOG textural features, and SkinBlobDetection or every other possible detection feature different from GCFused for post filtering. According to various exemplary embodiments ofthe invention one or more additional General Classification Features(GCF) are used to improve the accuracy of the initial screening.According to various exemplary embodiments of the invention the GCFsinclude one or more of color moment, Gabor function, color histogram,skin blob geometric information, color layout, intensity edge histogram,3 colors plane edge histogram, color structure and scalable color. Insome embodiments, each GCF is expressed as a vector with a naturalnumber value of 1 or 2 representing two class discrimination system andtwo probability variables between 0 and 1. In some embodiments, a globalprobability vector is used to summarize 2, 3, 4, 5, 6, 7, 8 or moreGCFs. Alternatively or additionally, in some embodiments a formula isused to summarize 2, 3, 4, 5, 6, 7, 8 or more GCFs and/or the globalprobability vector. In some embodiments various types of artificialintelligence to detect reduce data and/or detect content. For example, aConvolutional neural network (CNN) routine may be used (for exampleavailable routines may be used such as AlexNet, VGG, GoogLeNet orResNet).

In some embodiments, data may be added back at some points along theprocess. For example, after the first analysis for content some framesmay be sent for hand processing. For example, hand processing may not beslowed significantly by large resolution of individual frames.Optionally, in a case where a spatial resolution of a frame was reducedduring processing, the original high-resolution version of the frame maybe used for hand processing. FIG. 3 is a block diagram illustration of asystem of on-line video filtering in accordance with an embodiment ofthe current invention. In some embodiments, a user device 243 mayrequest a video file (e.g. from an Internet server 244). The server 244optionally retrieves the file from the Internet 260. Before deliveringthe file to user device, the sever optionally passes the file through anautomatic in-line filter 245. The file may be sent to the user onlyafter receiving approving from the filter. It may be desired to supplythe video content such that 1) the user does not have to wait longbefore receiving the first block of the video and/or 2) further blocksof the video are sent to the user within a time faster then the buffertime of the previous block (e.g. the video is screened smoothly withoutbreaks waiting for the next video block). In some embodiments, beforesending approval to the server 244, the filter 245 analyzes the video todetermine if it contains an undesired content. Optionally the filter 245includes modules to perform the analysis fast enough to deliver thevideo within the time limitations.

In some embodiments, the video may include a sequence of key framesand/or transition data, Decoding the checking the entire video may beprohibitively expensive in terms of computing time. Optionally the dataand/or complexity of the video may be reduced to a collection of samplespreserving the undesired content. Optionally analysis is performed on asummary of the video rather than decoding and/or checking the entire theentire video.

In some embodiments, a video disassembly module 247 may be supplied toextract the key frames from the video. Optionally, a collection offrames is sent from the disassembly module 247 to a video summary module246. For example, the video summary module 246 may remove frames thatare closer in time than a critical time interval (such as a REV and/or aportion thereof). Alternatively or additionally, the summary module maysearch for low information frames and/or remove them. Alternatively oradditionally, the summary module may remove redundant frames. Forexample, the summary module may use a measure of content and keep onlyone frame of a group of slides having a similar measure of content. Forexample, the measure of content may be represented by a vector and/or ascalar value. For example, an edge histogram EHD may be use as a measureof similarity. Optionally, redundant similar frames are removedregardless to their position on the video. Alternatively oradditionally, redundant similar frames are only eliminated if they areclose to each other in time in the video. Alternatively or additionally,eliminating similar slides accounts for the sequence of the frames. Insome embodiments the summary module will contain a content specificfilter. For example, a when searching for pornography, a filter mayeliminate slides that do not contain a lot of skin tones and/or havefull color (i.e. are not dark and/or are not dominated by one color[e.g. a picture taken under blue light may be dominated by blue and notinclude recognizable skin tones due to the source light]). Optionallythe summary module 246 outputs summary data to a detection module 248.

In some embodiments, the summary data is sent from the summary module toa detection module. Optionally the detection module 248 determineswhether the searched-for content (for example undesirable content) ispresent in the summary data. Alternatively or additionally, thedetection module 248 determines a probability that the searched-forcontent is present in the data set and/or in a frame and/or in a groupof frames. Optionally, the decision module 254 computes the probably ofthe presence of the content in the video and/or a level of uncertaintyand/or a way to reduce the uncertainty. Optionally, for example, if theuncertainty is too great, the decision module 254 will send a portion ofthe video to a further processing module 252 for further processing, forexample to reduce the uncertainty.

In some embodiments, data for further processing is further summarized.For example, further summarizing may include removing certain data tomake the further processing fast. Alternatively or additionally, thefurther summarizing may include restoring certain data that was removedin the first summarizing set. For example, in some embodiments, furtherprocessing module 252 may include manual classification group (forexample human viewers and/or viewing equipment). Optionally, one or moreframes are selected from the summary data for manual classification.Optionally, the selection of frames for further processing is based onthe level of uncertainty in each frame. Alternatively or additionally,selection of frames for further processing is based on the level ofuncertainty that each frame can reduce in the video. For example, insome cases when resolution was reduced to speed automatic processing,higher resolution frames (for example the original frames stored in amemory) are sent for further processing. Optionally, the results of thefurther processing are sent to the decision module 254. For example, thedecision may be based on the results of the detection module 248 and/orthe further summary module 254 (for example including an extendedprocessing center) instruct the server 244 whether to supply the videoto the user device 243.

In some embodiments, a user device and/or the further processing module252 may include a video decoding module and/or a user interface fordisplaying a video and/or a frame therefrom. Alternatively oradditionally, the user device 243 and/or the further processing module252 may include a user interface for communication of a human with thesystem. Alternatively or additionally, the user device 243 and/or theextended processing module 252 may include communication components forcommunicating with the server 244 and/or the decision module 254.

FIG. 4 in a flow chart overview of a method of detecting content inaccordance with an embodiment of the current invention. In someembodiment, an initial simple method is used to reduce content from thefull video. For example, resolution may be reduced 462. For example,steps of the reduction 462 of data may take advantage of inherentdivisions in the data, for example the division between key frames andtransition data. In some embodiments, a representative element isdetermined and resolution is reduced to a level at which and acceptablepreserving of information on the scale of the REV. Optionally,significant data will not be lost while the volume of the data issignificantly reduced.

In some embodiments, further reduction of the data set may employprogressively more complex routines. For example, the resolution reduceddata set may be processed to remove 236 low information data and/orremove 238 redundant data. Optionally, simple detection routines may beused to recognize and remove 240 non-candidate data.

In some embodiments, the reduced data set is analyzed 408 for thesearched-for content and/or an uncertainty is computed 410. Optionally,content may be found using an image analysis system, an example of sucha system is disclosed in U.S. Pat. No. 9,805,280 by the presentinventor. Optionally, further processing 414 may be used for example toreduce the uncertainty. Optionally, the data is further prepared 464 forfurther processing 414. For example, preparing 464 may include furtherreducing the data (e.g. by extracting only data that has highuncertainty and/or where reducing uncertainty will significantly reducethe uncertainty of the entire video) and/or only a portion of thesummary data set is sent for further processing 414. Optionally some ofthe original data is restored to the data set before further processing.

It is expected that during the life of a patent maturing from thisapplication many relevant robotic technologies and/or ArtificialIntelligence technologies will be developed and the scope of the termsare intended to include all such new technologies a priori.

As used herein the term “about” refers to ±10%

The terms “comprises”, “comprising”, “includes”, “including”, “having”and their conjugates mean “including but not limited to”.

The term “consisting of” means “including and limited to”.

The term “consisting essentially of” means that the composition, methodor structure may include additional ingredients, steps and/or parts, butonly if the additional ingredients, steps and/or parts do not materiallyalter the basic and novel characteristics of the claimed composition,method or structure.

As used herein, the singular form “a”, “an” and “the” include pluralreferences unless the context clearly dictates otherwise. For example,the term “a compound” or “at least one compound” may include a pluralityof compounds, including mixtures thereof.

Throughout this application, various embodiments of this invention maybe presented in a range format. It should be understood that thedescription in range format is merely for convenience and brevity andshould not be construed as an inflexible limitation on the scope of theinvention. Accordingly, the description of a range should be consideredto have specifically disclosed all the possible subranges as well asindividual numerical values within that range. For example, descriptionof a range such as from 1 to 6 should be considered to have specificallydisclosed subranges such as from 1 to 3, from 1 to 4, from 1 to 5, from2 to 4, from 2 to 6, from 3 to 6 etc., as well as individual numberswithin that range, for example, 1, 2, 3, 4, 5, and 6. This appliesregardless of the breadth of the range.

Whenever a numerical range is indicated herein, it is meant to includeany cited numeral (fractional or integral) within the indicated range.The phrases “ranging/ranges between” a first indicate number and asecond indicate number and “ranging/ranges from” a first indicate number“to” a second indicate number are used herein interchangeably and aremeant to include the first and second indicated numbers and all thefractional and integral numerals therebetween.

It is appreciated that certain features of the invention, which are, forclarity, described in the context of separate embodiments, may also beprovided in combination in a single embodiment. Conversely, variousfeatures of the invention, which are, for brevity, described in thecontext of a single embodiment, may also be provided separately or inany suitable subcombination or as suitable in any other describedembodiment of the invention. Certain features described in the contextof various embodiments are not to be considered essential features ofthose embodiments, unless the embodiment is inoperative without thoseelements.

Although the invention has been described in conjunction with specificembodiments thereof, it is evident that many alternatives, modificationsand variations will be apparent to those skilled in the art.Accordingly, it is intended to embrace all such alternatives,modifications and variations that fall within the spirit and broad scopeof the appended claims.

All publications, patents and patent applications mentioned in thisspecification are herein incorporated in their entirety by referenceinto the specification, to the same extent as if each individualpublication, patent or patent application was specifically andindividually indicated to be incorporated herein by reference. Inaddition, citation or identification of any reference in thisapplication shall not be construed as an admission that such referenceis available as prior art to the present invention. To the extent thatsection headings are used, they should not be construed as necessarilylimiting.

What is claimed is:
 1. A computer implemented method for fastdetermination of a presence of a particular content in video comprising:summarizing a segment of the video by reducing temporal resolution insaid segment to form a summary; detecting in said summary a relatedcontent associated with said particular content; eliminating portionswherein said related content was not detected in said detecting fromsaid summary thereby producing a reduced summary; analyzing said reducedsummary to detect said particular content by means of an artificialintelligence routine.
 2. The computer implemented method of claim 1,wherein said artificial intelligence routine is trained to estimate aprobability of the presence of the particular content.
 3. The computerimplemented method of claim 1, wherein said summarizing includesextracting a collection of key frames from said segment.
 4. The methodof claim 3, wherein said summarizing includes reducing said collectionof key frames including: identifying duplicate frames in said collectionof key frames; removing said duplicate frames from said collection ofkey frames resulting in a set of unique frames.
 5. The computerimplemented method of claim 4, wherein said detecting includesidentifying in said set of unique frames, by means of a contentdependent routine, frames which include a said related content.
 6. Thecomputer implemented method of claim 5, further wherein said eliminatingincludes eliminating those frames wherein said related content was notdetected from said set of unique frames to produce a video summaryhaving frames including said related content.
 7. The computerimplemented method of claim 1, wherein said reducing further includes:determining a representative element volume (REV) of said particularcontent and reducing a level of precision to an order of said REV. 8.The computer implemented method of claim 3, wherein said reducingincludes removing a low information key frame from said collection ofkey frames.
 9. The computer implemented method of claim 8, whereinremoving low information key frame includes computing an entropy of akey frame of said video and removing said key frame of the video whensaid entropy is low.
 10. The computer implemented method of claim 1,wherein said detecting and said eliminating are applied withoutaccounting for sequencing within the video.
 11. The computer implementedmethod of claim 1, wherein probability that the particular content ispresent in the video takes into account a spatial autocovariance. 12.The computer implemented method of claim 1, further comprisingperforming further processing to reduce an uncertainty in said presenceof the particular content in the video.
 13. The computer implementedmethod of claim 12, wherein said further processing includes manualanalysis of a frame.
 14. The computer implemented method of claim 1,wherein said particular content is preselected.
 15. The computerimplemented method claim 14, wherein said particular content is anundesired content.